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Evaluafcins Outcones of Competency- 
Based Teacher Education 

James R. Okey and Donald W, Humphreys 
Indiana University 

A basic assumption underlying teacher education programs (and 
cxpccially competency-based programs) is that there are teachers' be- 
haviors that can be taught, learned and used that affect what pupils do.' 
To help educators plan training programs for teachers with some confidence 
in the behaviors or skills selected for study, research is needed that 
directly links teacher behaviors with pupil outcomes. 

Studies relating teacher behaviors to pupil accomplishments are 
called process-product studies (Rosenshine and Furst, 1971). The studies 
can be either correlational or experimental, T^at a teacher does 
(process) is correlated with what pupils do (product) or the effect of 
the process variable on the product variable is measured in an eicperi-^ 
mental study. - . ^ 

The importance of process-product research is obvious. Unless 
direct relationships between teacher and pupil behaviors can be established 
the credibility of teacher education is weakened. 

Question asking is a teaching skill area that receives considerable 
attention. Studies consistently show a) that teachers ask an enormous 
number of questions and b) that the, questions require primarily memorized 
responses from students (cf. Floyd, 1968). 

\ 

Paper presented at the Aiaerican Educational Research Association Annual 
Meeting, Chicago, April, 1974. 

The activities reported in this paper wore supported by the National 
Center for the Devclopnent of Training Materials in Teacher Education 
at Indiana University under a grant from the U.S. Office of Education. 
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The purpose of this study was to investigate the effect of teacher 
questioning behavior (the process) on pupil cognitive achievement (the 
product) r The intention was to train teachers to use questions from each 
of the six levels of Bloom's cognitive Taxonomy (1956) during classroom^ 
discussions and to determine the effect this would have on pupil achieve- 
ment of test questions from each of the sane levels. 

Procedure 

Sixteen inservice fourth and fifth grade teachers'^ were selected 
for the study and randomely assigned to treatment and control groups. 
Each teacher made an audiotape of a 15 to 20 minute question and answer 
discussion to provide a baseline measure of questioning behaviors. The 
16 teachers also completed two pretests to measure their ability to 
classify and to wite questions for each of the six Taxonomy levels • 

After gathering baseline data, tl)e 8 teachers in the experimental 
group studied exercises designed to alter their question asking practices. 
The exercises included a tape/slide program on the levels of the 
Taxonomy and a self -instructional manual that provided instruction in 
classifying, vrriting, and sequencing questions (Okey, Humphreys and 
Bedwell, 1973). 

The instruction on question asking included about 5 hours of both 
Individual and group work. Some of this time was spent practicing with 
pupils, recording these sessions an audio tape and classifying the 
questions used. Thus the teachers were encouraged to practice the desired 
behaviors and to provide their o\m feefiback. 

*The asslstan6e_.of Glenn' Linnert, science consultant, and the 16 teachers 
from the New Albany-Floyd County (Indiana) School Corporation is gratefully 
acknowledged. 
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Folloving completion of instruction by the 8 e;.TperlinentaI group 
teachers, all 16 teachers in the study taught 3 lessons to the pupils 
in their classes. For each lesson the pupils read a science article 
from 5^ book regularly used in the school and then participated in 
ques'ilon and answer/discussion session led by their teacher. Each ^ 
teacher made an audio tape of each (.session. Teachers in the treatment 
group were told to vary their questioning during the discussion to^ 
Include questions from all 6 Taxonomy levels. Control teachers were 
told to use whatever questions they thought were appropriate. After 
the discussion, pupils in all classes coiapleted a short multiple-choice 
and open-^response test that included questions from each level of the - 
Taxonomy . . ^ 



Results 



Table 1 summarizes the major findings from the study • ' The teachers 
In the experimental and control groups were siraila^ to one another before- 
the treatment began in their ability to classify and compose questions 
and in the range of questions they asked during Lesson 1. There were 
also remarkably similar after instruction. Some alterations were made 
In the treatment teachers abilities but the results are in no ^^y im- 
pressive. D(;spite the training, the treatment teachers did not significantly 
alter the number of high level question^ they used with their pupils* 
As expected under these circumstances, pupil achievement was not signifi- 
cantly different for experimental and control teachers. 

Teachers 'in the treatment group also completed a questionnaire follow- 
ing their study of the question asking training materials. Results from 
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this survey and informal observations made during the training sessions 
and classroom visits showed high enthusiasiit and interest among the treat- 
ment teachers, They^ liked the training, perceived it as important, and 

'■\ ■ ... ■ • ' 

thought they were changing thfir classroom* behaviors. Despite these 

' ^ ■ ' • ■ ^ ^ ' ' ' 

posit:^ve fellings, analysis of the audio tapes failed to sho^ differences 

. ' * ' ' ' \ 

in the question asking'' practices of treatment and control teachers. 
Discussion ' ■ ■ jf 



A maj' ' point to be made is- that the experiment does n^t show that 
question asking skj^.lls have no relatioii to pupjl achievement. Treatment 
group teachers did not use significantly more high level questions after 
their training so the hypothesis relating teacher questions to pupil 
achievement was never tested. \ ^ 

Despite not getting' a test of the research hypothesis, the study 
was valuable. Information was obtained that helped to revise the in- ^ 
structional. materials on question asking. '''f Perhaps more important, 
knowledge of how (and how not) to conduct a process-product study «as < 
gained. , 

Process-product studies relating teacher behavior to pupil outcomes 
are often conducted in two parts or phases, The first phase 'involves 
, the acquisition of a skill (or skills) by teachers and the ^^econd phase * 
the effectiveness of the skills when used. Figure 1 shows .the two phases 
and the Independent and dependent. variables In ea^h portion of the 
process-product study, ' ^ 



*The revised verson o^ the m^aferials used to train teachers- to- ask highe^ 
level questions is now being tested. , 



SKILL ACQUISITION SKILL EFFECTIVENESS 









lUPEPEITDENT ' 


teacher training materials 


use of sl:lll(s) by 


VARXABLE 


and procedures 


ceacner 


DEPENDENT 


use of skill (s>^by 


cognitive and/or affective 


VARIABLE 


teachers 


pupil outcdmes 



Jigure 1. Two phases of a process-product study. 



The dependent variable (use of skills by teachers) during the skill 
acquisition phase becomes the i*ndependent variable for the skilj. effec- 
tiveness phase of the study. Using this conception of process-product 
research ^kes the defects in this s~tudy obvious; the independent variable 
Was never put into effect in the skill effectiveness phase because the 
teachers didn't learn the skills in the acquisition phase of the experiment. 

Process-product studies may include both a skill acquisition and skill 

effectiveness phase but only the latter is crucial to the testing of 

relationships between teacher behaviors (products) and pupil outcomes 

(processes). Thus studies can be planried using Various dombinations of / 

the tvo phases. Some possibilities are: 

Conduct a skill acquisition study with treatment and control 
teachers (some teachers study a skill and other don't). Then 
contrast the effects of the traiited and untrained teachers in 
the second yCef feet iveness) phase. 

• *, ' • 

2. Train a group of teachers to use certain skills. Then have them 
deliberately use and not use the skills with different groups 

of students to determine the effects. Ko control group is. pre- 
sent during the acquisition phase. Teachers seifve as their o\m 
conticols during the effectiveness phase of the ^tudy. Okey 
and Ciesla ' (1973) report a study using this procedure. 

3. Identify teachers that can already exhibit the teaching' skills 
of concern. Compare their effectiveness v;ith a selected control 
group or have them use and not use; the skills as described above. 

/ This procedure eliminates ^ the skill' acquisition phase of the 
prociq^s-product study and Concentrates only on the effectiveness 
phase. ^ 



It Is essential In process-product studies to demonstrate that 
teachers actually use the skills being strudled* If this Is not done, 
there is no way to tell if the independent variable is operating in 
the effectiveness phase of the study and to show that the process-product 
hypothesis has actually been tested. Worthen (1953) and Ciesla and 
Okey (1974) describe procedures and instruments, for determining whether 
teachers are using the skills of interest. 

Because of the failure to -train teachers in the way we had intended 
In this study, several .changes are being made in both bur training and 
research procedures. One of these is to more clearly separate the skill 
acquisition and skill effectiveness phases of the process-product 
studies. There is no point in conducting an effectiveness study if 
skill acquisition has not occurred. 

Another change t/e have made is to rely much less on teacher self- 
evaluation during skill acquisition. At least some teachers are unable 
to tell when they have acquired a skill and need another person to assist 
in evaluation. Peck and Tucker (1973) conclude that teacher self- 

r 

evaluation Is less effective than when an outsider participates in the 
feedback process. We are presently using both other teachers and super- - 
visors to provide feedback to teachers on their ability to use a skill. 
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